Skip to content

feat: serializers module for RDB encode/decode#382

Merged
AviAvni merged 1 commit intomainfrom
serializers-module
Apr 14, 2026
Merged

feat: serializers module for RDB encode/decode#382
AviAvni merged 1 commit intomainfrom
serializers-module

Conversation

@AviAvni
Copy link
Copy Markdown
Contributor

@AviAvni AviAvni commented Apr 14, 2026

Summary

  • buffered_io.rs: BufferedWriter/BufferedReader with 256KB chunks and type-tagged framing (C FalkorDB v2 compatible)
  • encoder/mod.rs: rdb_save_graph_key encodes graph header, schema, matrices, attrs as chunked payloads
  • decoder/mod.rs: rdb_load_graph decodes v19 format, reconstructs Graph via Graph::restore()
  • mod.rs: DECODE_STATE, VKEY_STATE globals, build_multi_key_payloads helper, encoding version constants

Module is not yet wired into lib.rs — that happens in the final RDB persistence PR.

Part of splitting #359 into smaller reviewable PRs.

Test plan

  • cargo check passes (module compiles but is not yet reachable)
  • Full integration tested in follow-up PR when wired into redis_type.rs

Summary by CodeRabbit

Release Notes

  • New Features
    • Added graph RDB serialization support using v19 format for persistent storage
    • Introduced buffered IO layer for optimized data serialization and deserialization operations

- buffered_io.rs: BufferedWriter/BufferedReader with 256KB chunks and type-tagged values
- encoder/mod.rs: rdb_save_graph_key encodes graph header, schema, matrices, attrs
- decoder/mod.rs: rdb_load_graph decodes v19 format, reconstructs Graph via restore()
- mod.rs: DECODE_STATE, VKEY_STATE, build_multi_key_payloads, encoding version constants
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 14, 2026

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Four new Rust modules establish RDB serialization for graph data in v19 format. The changes introduce a buffered IO layer for type-tagged binary reads/writes, graph encoder/decoder implementations with multi-key support, and global state management for coordinating single and multi-key graph persistence operations.

Changes

Cohort / File(s) Summary
Buffered IO Layer
src/serializers/buffered_io.rs
Implements BufferedWriter (accumulates tagged records in 256KB buffer, flushes via Redis) and BufferedReader (loads chunks, parses type tags). Includes Writer and Reader trait impls. Note: Trait implementations contain self-recursive calls requiring attention.
Serialization Infrastructure
src/serializers/mod.rs
Defines core types (Header, Schema, EncodeState, PayloadEntry), implements Encode<19> and Decode<19> traits, exports global state (VKEY_STATE, DECODE_STATE) for multi-key coordination and index definitions.
Graph Decoder
src/serializers/decoder/mod.rs
Implements rdb_load_graph for v19 decoding with single and multi-key support, payload dispatch by EncodeState, PendingGraph state machine for multi-key coordination, and finalize_pending_graph to reconstruct graphs with indexes.
Graph Encoder
src/serializers/encoder/mod.rs
Implements rdb_save_graph entry point, rdb_save_graph_key for header/schema/payload encoding, and build_multi_key_payloads to distribute graph entities across RDB keys with fixed matrix entries on key 0.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Encoder as Graph Encoder
    participant Writer as BufferedWriter
    participant Redis as RDB Storage
    
    Client->>Encoder: rdb_save_graph(graph)
    Encoder->>Encoder: Build payload directory
    Encoder->>Writer: new(rdb_io)
    Encoder->>Writer: write_unsigned (header fields)
    Encoder->>Writer: write_buffer (schema)
    Encoder->>Writer: write_unsigned/write_buffer (payloads)
    loop For each payload
        Encoder->>Encoder: encode_payload(graph)
        Encoder->>Writer: write_unsigned/write_buffer
    end
    Writer-->>Redis: flush on buffer overflow (256KB)
    Encoder->>Writer: finish()
    Writer-->>Redis: final chunk
    Encoder-->>Client: Done
Loading
sequenceDiagram
    participant Client
    participant Decoder as Graph Decoder
    participant Reader as BufferedReader
    participant Redis as RDB Storage
    participant State as DECODE_STATE
    
    Client->>Decoder: rdb_load_graph(rdb)
    Redis-->>Reader: load_string_buffer (chunk 0)
    Reader->>Decoder: parse Header/Schema
    
    alt Single Key (key_count == 1)
        Decoder->>Reader: read payloads
        Decoder->>Decoder: decode_payloads (to local stores)
        Decoder->>Decoder: finalize (commit attrs, rebuild indexes)
        Decoder-->>Client: return Graph
    else Multi Key (key_count > 1)
        Decoder->>State: init PendingGraph (key 0)
        Decoder->>Reader: read/decode payloads
        Decoder->>State: store in pending_graph
        State-->>Decoder: keys_remaining--
        Decoder->>Client: return None
        Client->>Decoder: rdb_load_graph(key N)
        Decoder->>State: load PendingGraph
        Decoder->>Reader: read/decode payloads
        Decoder->>State: merge into pending_graph
        Note over State: When keys_remaining == 0
        Decoder->>Decoder: finalize_pending_graph()
        Decoder->>State: store finalized Graph
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Poem

🐰 Hops through buffered streams so bright,
Type-tagged records, buffered right,
Multi-keys dance across the store,
Graphs persist forevermore!
RDB v19, our finest lore! 🎉

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 64.15% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: serializers module for RDB encode/decode' accurately captures the main change: adding a new serializers module with RDB encoding and decoding functionality.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch serializers-module

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@AviAvni AviAvni merged commit a60b153 into main Apr 14, 2026
12 of 14 checks passed
@AviAvni AviAvni deleted the serializers-module branch April 14, 2026 14:14
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.33%. Comparing base (7c72209) to head (5612acf).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #382      +/-   ##
==========================================
+ Coverage   82.32%   82.33%   +0.01%     
==========================================
  Files         110      110              
  Lines       30078    30078              
==========================================
+ Hits        24762    24766       +4     
+ Misses       5316     5312       -4     
Flag Coverage Δ
fuzz 40.61% <ø> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant